International  Journal  of  Electronics, 
Communication  & Instrumentation  Engineering 
Research  and  Development  (UECIERD) 
ISSN(P):  2249-684X;  ISSN(E):  2249-7951 
Vol.  5,  Issue  5,  Oct  2015, 1-10 
© TJPRC  Pvt.  Ltd. 


TRAMS 

STELLAR 

•Journal  Publications  • Research  Consultancy 


Most  Trusted  | Most  Subscribed  to  | Most  Referred  to 


“MODELLING  OF  COMPLEX  CHAOTIC  SIGNALS  USING  ARTIFICIAINTELLIGENCE” 

APURVA  N.  SHENDE1  & SANJAY  L BADJATE2 

'S.  B.  Jain  Institute  of  Technology,  Management  and  Research,  Nagpur,  India 
2Vice  Principal,  S.  B.  Jain  Institute  of  Technology,  Management  and  Research,  Nagpur,  India 


ABSTRACT 

In  this  project  we  are  considering  different  Chaotic  signals  and  their  results  so  that  we  can  work  on  the  best 
method.  Here  Chaos  (Irregular  motions  ) are  used.  Filtering  has  been  invoked  to  reduce  the  obviously  noisy  character  of 
the  monthly  sunspot  numbers.  However,  here  shown  that  straight  for  wardlinear  filtering  can  also  make  a stochastic  signal 
appear  to  below-dimensional,  so  one  cannot  eliminate  the  possibility  that  the  sunspot  numbers  are  the  result  of  a stochastic 
process  with  a “noisily  periodic”  component.  We  are  comparing  numerational  knowledge  based  techniques  for  forecasting 
has  been  proved  highly  successful  in  present  time.  The  purpose  of  this  paper  is  to  examine  the  effects  of  several  important 
neural  network  factors  on  model  fitting  and  forecasting  the  behaviors. 

KEYWORDS:  Artificial  Neural  Network  (ANN),  Data,  Prediction,  Forecasting,  Foreign  Exchange  Rate,  Autoregressive 
Integrated  Moving  Average  (ARIMA) 

INTRODUCTION 

We  investigate  the  effectiveness  of  connectionist  architectures  for  predicting  the  future  behavior  of  nonlinear 
dynamical  systems.  We  stress  on  real-world  time  series  of  limited  record  length.  Examples  are  analyzed:  sunspot  series  and 
chaotic  data  from  a computational  ecosystem  is  the  benchmark.  The  problem  of  over  fitting,  particularly  serious  for  short 
records  of  noisy  data,  is  addressed  both  by  using  the  statistical  method  of  validation  and  by  adding  a complexity  term  to  the 
cost  function  ("back-propagation  with  weight-elimination").  The  dimension  of  the  dynamics  underlying  the  time  series,  its 
Liapunov  coefficient,  and  its  nonlinearity  can  be  determined  via  the  network.  We  also  show  superiority  of  radial  basis 
functions  for  high-dimensional  input  spaces.  As,  since  the  ultimate  goal  is  accuracy  in  the  prediction,  we  find  that  sigmoid 
networks  trained  with  the  weight-elimination  algorithm  outperform  traditional  nonlinear  statistical  approaches.  The 
Artificial  Neural  Networks  have  received  increasing  attention  as  decision-making  tools. 

Artificial  Neural  Network 

The  electronic  models  of  ANN  are  based  on  neural  structure  of  the  brain.  One  is  ‘artificial  neurons’.  These  are 
called  artificial  neural  networks  (ANNs).  An  artificial  neuron  is  a computational  model  inspired  in  the  natural  neurons. 
When  the  signals  received  are  strong  enough  i.  e.  surpass  a certain  threshold,  the  neuron  is  activated  and  emits  a signal 
though  the  axon.  This  signal  might  be  sent  to  another  synapse,  and  might  activate  other  neurons.  Natural  neurons  receive 
signals  through  synapses  located  on  the  dendrites  or  membrane  of  the  neuron 
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Figure  1:  Natural  Neurons  (Artist’s  Conception) 

Next  function  computes  the  output  of  the  artificial  neuron  (in  dependence  of  a certain  threshold).  For  processing 
ANNs  combine  artificial. 
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Figure  2:  An  Artificial  Neuron 

If  the  neuron  have  higher  weight,  the  stronger  the  input  which  is  multiplied  by  it  will  be.  Negative  weights  are 
also  there,  then  the  signal  is  inhibited  by  the  negative  weight.  Depending  on  the  weights,  the  computation  of  the  neuron 
will  be  different.  We  adjust  the  weights  of  an  artificial  neuron,  we  can  obtain  the  output  we  want  for  specific  inputs.  But 
when  we  have  an  ANN  of  hundreds  or  thousands  of  neurons,  it  would  be  quite  complicated  to  find  by  hand  all  the 
necessary  weights.  Using  algorithms  which  can  adjust  the  weights  of  the  ANN  in  order  to  obtain  the  desired  output  from 
the  network.  Learning  or  training  is  this  process  of  adjusting  the  weights. 

Here  we  study  behaviour  and  control  in  animals  and  machines,  but  also  there  are  ANNs  which  are  used  for 
engineering  purposes,  such  as  pattern  recognition,  forecasting,  signal  processing  and  data  compression. 

LEARNING  IN  ARTIFICIAL  NEURAL  NETWORKS 

Ability  to  learn  is  one  of  the  most  impressive  features  of  artificial  neural  networks.  And  is  the  biological  nervous 

system. 

Learning  Types 

The  different  algorithms  that  can  be  used  when  training  artificial  neural  networks,  with  their  own  specific 
advantages  and  disadvantages.  The  learning  process  within  artificial  neural  networks  is  a result  of  altering  the  network's 
weights,  with  some  kind  of  learning  algorithm  used  there.  The  objective  is  to  find  a set  of  weight  matrices  involved  there 
which  when  applied  to  the  network  should  - hopefully  - map  any  input  to  a correct  output  in  last  stage.  Here  we  focus  on 
supervised  learning.  But  before  we  begin,  lets  take  a quick  look  at  the  three  major  learning  paradigms. 

Supervised  Learning 

It  is  possible  to  calculate  an  error  based  on  it's  target  output  and  actual  output  by  providing  the  neural  network 
with  both  an  input  and  output  pair.  And  the  error  is  used  to  make  corrections  to  the  network  by  updating  it's  weights. 
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Unsupervised  Learning 

This  type  of  learning  is  used  by  many  recommendation  algorithms  because  of  their  ability  to  predict  a user's 
preferences  based  on  the  preferences  of  other  similar  users  it  has  grouped  together. 

Reinforcement  Learning 

Reinforcement  learning  is  similar  to  supervised  learning,  instead  of  providing  a target  output  a reward  is  given 
based  on  how  well  the  system  performed  in  different  conditions.  It  is  trial-and-error  process.  This  paradigm  relates  strongly 
with  how  learning  works  in  nature. 

ANN 

Artificial  neural  networks  (ANNs)  and  auto-regressive  integrated  moving  average  (ARIMA)  models  for  time 
series  forecasting  are  briefly  reviewed. 

The  ANN  Approach  to  Time  Series  Modeling 

The  network  model  is  largely  determined  by  the  characteristicsof  the  data.  Single  hidden  layer  feed  forward 
network  is  the  most  widely  used  model  form  for  time  series  modeling  and  forecasting. 

The  Auto-Regressive  Integrated  Moving  Average  Models 

Here  the  data  transformation  is  often  required  to  make  the  time  series  stationary.  Stationarity  is  a necessary 
condition  in  building  an  ARIMA  model  used  for  forecasting.  A stationary  time  series  is  characterized  by  statistical 
characteristics  such  as  the  mean  and  the  autocorrelation  structure  being  constant  overtime.  When  the  observed  time  series 
presents  trend  and  heteroscedasticity,  differencing  and  power  transformation  are  applied  to  the  data  to  remove  the  trend 
and  to  stabilize  the  variance  before  an  ARIMA  model  can  be  fitted. 

ARTIFICIAL  NEURAL  NETWORK  FOR  TIME  SERIES  FORECASTING 

Issues  in  ANN  Modeling  for  Forecasting 

ANNs  have  many  satisfactory  characteristics,  building  a neural  network  forecaster  for  a particular  forecasting 
problem  is  a nontrivial  task.  So  the  modeling  should  be  done  carefully.  One  critical  decision  is  to  determine  the  appropriate 
architecture,  that  is,  the  number  of  layers,  the  number  of  nodes  in  each  layer,  and  the  number  of  arcs  which  interconnect 
with  the  nodes.  Different  network  design  decisions  include  the  selection  of  activation  functions  of  the  hidden  and  output 
nodes,  the  training  algorithm,  data  transformation  or  normalization  methods,  training  and  test  sets,  and  performance 
measures. 

Proposed  Design  Comparison 

From  [1]  The  number  of  hidden  nodes  allows  neural  networks  to  capture  nonlinear  patterns  and  detect  complex 
relationships  in  the  data.  We  experiment  with  a relatively  large  number  of  input  nodes.  There  is  no  upper  limit  on  the 
possible  number  of  hidden  nodes  in  theory.  However,  it  is  rarely  seen  in  the  literature  that  the  number  of  hidden  nodes  is 
more  than  doubling  the  number  of  input  nodes.  The  combination  of  ten  input  nodes  and  five  hidden  nodes  yields  a total  of 
50  neural  network  architectures  being  considered  for  each  insample  training  data  set.  A comprehensive  study  of  neural 
network  time  series  forecasting,  finds  that  neural  network  models  do  not  necessarily  require  large  data  set  to  perform  well. 
Three-layer  feed  forward  neural  networks  are  used  to  forecast  the  Indian  Rupee  (INR)  / US  Dollar  (USD)  exchange  rate. 
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Logistic  activation  functions  are  employed  in  the  hidden  layer  and  the  linear  activation  function  is  utilized  in  the  output 
layer. 

From  [2]  Data  Sets:  The  basic  data  set  is  the  monthly  mean  Wolf  sunspot  numbers,  hereinafter  referred  to  as  the 
“raw”  sunspot  numbers.  The  analytical  methods  are  data  intensive,  forcing  us  to  use  all  values.  From  the  raw  sunspot 
numbers,  we  can  construct  a number  of  other  data  sets.  First,  one  makes  the  smoothed  monthly  sunspot  numbers  via  a 
centered  12-month  average  we  refer  to  this  set  as  the  “smoothed”sunspot  numbers.  Consequently,  we  make  the 
“unrectified”  sunspot  numberset  by  (essentially)  alternating  the  signs  of  the  sunspot  numbers  each  solar  cycle.  The  detailed 
process  is  as  follows: first  they  find  the  minima  of  the  smoothed  monthly  sunspot  number  set.  Those  minima  become  our 
first  guess  for  thetimes  where  the  sign  of  the  sunspot  numbers  changes  (theantirectification  points).  A trial  unrectified  set  is 
made,  andthat  set  is  then  smoothed  via  the  same  centered  12-monthaveraging  process.  The  zero  crossings  of  the  smoothed 
setare  the  second  guess  for  the  antirectification  points.  A new  trial  unrectified  set  is  made  from  the  original  raw 
sunspotnumber  set;  and  this  process  is  repeated  until  the  set  of  antirectification  points  is  stable  (four  iterations).  Hence,  on 
the  basis  of  both  thephysical  magnetic  reversals  and  the  spectral  evidence,  we  suggest  that  the  unrectified  sunspot  number 
set  is  actually  the  fundamental  data  set.  From  the  unrectified  sunspot  numbers,  we  make  the  “smoothed  unrectified” 
sunspotnumbers  via  the  centered  12-month  average. 

From  [3]  The  exchange  rates  between  the  Indian  Rupee  (INR)  / US  Dollar  (USD)  are  obtained  from  Datastream 
International.  These  papers  examine  the  effects  of  several  factors  on  the  in  sample  fit  and  out  of  sample  forecasting 
capabilities  of  neural  networks.  The  neural  network  factors  investigated  are  the  number  of  inputs  and  hidden  nodes  which 
are  two  critical  parameters  in  the  design  of  a neural  network.  The  number  of  input  nodes  is  perhaps  the  most  important 
factor  in  neural  network  analysis  of  a time  series  since  it  corresponds  to  the  number  of  past  lagged  observations  related  to 
future  values.  It  also  plays  a role  in  determining  the  autocorrelation  structure  of  a time  series.  The  number  of  hidden  nodes 
allows  neural  networks  to  capturenonlinear  patterns  and  detect  complex  relationships  in  the  data.  We  experiment  with  a 
relatively  large  number  of  input  nodes.  There  is  no  upper  limit  on  thepossible  number  of  hidden  nodes  in  theory.  However, 
it  is  rarely  seen  in  the  literature  that  the  number  of  hidden  nodes  is  more  than  doubling  the  number  of  input  nodes.  In 
addition,  previous  research  indicates  that  the  forecasting  performance  of  neural  networks  is  not  as  sensitive  tothe  number 
of  hidden  nodes  as  it  is  to  the  number  of  input  nodes.  Thus,  five  levels  of  hiddennodes,  6,  12,  18,  24  and  30  will  be 
experimented. 

The  combination  of  ten  input  nodes  and  fivehidden  nodes  yields  a total  of  50  neural  network  architectures  being 
considered  for  each  insampletraining  data  set.  A comprehensive  study  of  neural  network  time  series  forecasting,  findsthat 
neural  network  models  do  not  necessarily  require  large  data  set  to  perform  well.  To  testif  there  is  a significant  difference 
between  large  and  small  training  samples  in  modeling  andforecasting  exchange  rates,  we  use  two  training  sample  sizes  in 
our  study.  The  large  sampleconsists  of  1043  observations  from  1989  to  2009  and  the  small  one  includes  365  data  points 
from2003  to  2009.  The  test  sample  for  both  cases  is  the  2010  data  which  has  52  observations.  The  random  walk  model  will 
be  used  as  a benchmark  for  comparison.  The  random  walk  is  a one-step  aheadforecasting  model  since  it  uses  the  current 
observation  to  predict  the  next  one  Three-layer  feed  forward  neural  networks  are  used  to  forecast  the  Indian  Rupee  (INR)  / 
US  Dollar(USD)  exchange  rate.  Logistic  activation  functions  are  employed  in  the  hidden  layer  and  the  linear  activation 
function  is  utilized  in  the  output  layer.  We  are  interested  in  one- step-ahead  forecasts,  one  output  node  are  deployed  in  the 
output  layer.  The  of  using  of  direct  optimization  procedure  in  neural  network  training.  To  be  more  certain  of  getting  the 
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true  global  optima,  acommon  practice  is  to  solve  the  problem  using  a number  of  randomly  generated  initial  solutions.  We 
train  each  network  50  times  by  using  50  sets  of  different  initial  arc  weights.  The  best  solutionamong  50  runs  is  used  as  the 
optimal  neural  network  training  solution. 

RESULTS 

The  plots  between  the  desired  output  and  actual  neural  network  output  are  plotted  for  these  three  models  for 
short-term  (1,  6 and  12)  and  long-term  (18  and  24)  months  ahead  prediction  on  testing  data  set.  By  the  close  visual 
inspection  of  the  plots  shown  in  Figure.  3a  for  FTLRNN  model,  Figure.  3b  for  MLPNN  and  Figure. 3c  for  SOFMNN 
model  for  1 -month  ahead  prediction,  the  focussed  time  lagged  recurrent  neural  network  (FTLRNN)  model  closely  follows 
the  actual  output  as  compared  to  the  static  MLPNN  and  SOFMNN  model. 


Figure  3:  a Desired  and  FTLRNN  Outputs  for  1-Month  Ahead  Prediction 


Figure  3:  b Desired  and  MLPNN  Outputs  for  1-Month  Ahead  Prediction 


Figure.3:  c Desired  and  SOFMNN  Outputs  for  1-Month  Ahead  Prediction 

Similarly,  the  plot  between  desired  output  and  actual  neural  network  output  are  plotted  for  6-month  ahead 
prediction  shown  in  Figure  4a  for  FTLRNN,  Figure  4b  for  MLPNN  model  and  Figure  4c  for  SOFMNN  model.  By  close 
visual  inspection  of  the  Figure  4a,  the  FTLRNN  predictor  closely  follows  the  desired  output.  The  MLPNN  and  SOFMNN 
are  failed  to  follow  the  desired  output  well  as  seen  from  the  plot  Figure  5b  and  Figure.  5c. 
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Figure  4:  a.  Desired  and  FTLRNN  Outputs  for  6-Month  Ahead  Prediction 


Figure  4:  b Desired  and  MLPNN  Outputs  for  6-Month  Ahead  Prediction 


Figure  4:  c Desired  and  SOFMNN  Outputs  for  6-Month  Ahead  Prediction 

Then  for  12-month  ahead  prediction,  the  plot  between  desired  and  actual  output  are  plotted  as  demonstrated  in 
Figure  5a  for  FTLRNN,  Figure  5b  for  MLPNN  and  Figure  5c  for  SOFMNN  predictor.  It  should  be  noted  that  the  FTLRNN 
output  is  deviating  slightly  the  desired  output  but  far  better  than  the  MLPNN  and  SOFMNN. 


Figure  5:  a Desired  and  FTLRNN  Outputs  for  12-Month  Ahead  Prediction 
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Figure  5:  b Desired  and  MLPNN  Outputs  for  12-Month  ahead  Prediction 


Figure  5:  c Desired  and  SOFMNN  Outputs  for  12-Month  Ahead  Prediction 

Next  for  18-month  ahead  prediction  the  plot  between  desired  output  and  actual  output  is  shown  in  Fig.4a  for 
FTLRNN,  Fig.  4b  for  MLPNN  and  4c  for  SOFMNN  model.  It  is  also  noted  that  the  output  of  FTLRNN  predictor  is 
deviating  marginally  from  the  desired  output.  The  MLPNN  and  SOFMNN  predictors  failed  to  follow  the  desired  output. 


Figure  6:  a Desired  and  FTLRNN  Outputs  for  18-Month  Ahead  Prediction 


Figure  6:  b Desired  and  MLPNN  Outputs  for  18-Month  Ahead  Prediction 


Figure  6:  c Desired  and  SOFMNN  Outputs  for  18-Month  Ahead  Prediction 

Similarly,  for  24-month  ahead  prediction,  the  plot  between  desired  output  and  actual  output  are  plotted  as 
illustrated  in  Fig. 6a  for  FTLRNN  predictor,  in  Fig. 6b  for  MLPNN  predictor  and  in  Fig. 6c  for  SOFMNN  NN  predictor.  It  is 
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clear  that  the  FTLRNN  output  followed  the  desired  output.  The  MLPNN  and  SOFMNN  output  failed  to  follow  the  desired 
output  completely. 


Figure  7:  a Desired  and  FTLRNN  Outputs  for  24-Month  Ahead  Prediction 


Figure  7:  b Desired  and  MLPNN  outputs  for  24-month  Ahead  prediction 


Figure  7:  c Desired  and  SOFMNN  Outputs  for  24-Month  Ahead  Prediction 

Thus  FTLRNN  model  has  learned  the  dynamics  of  the  complex  monthly  sunspot  chaotic  time  series  elegantly 
well.  On  the  contrary,  the  MLPNN  and  SOFMNN  model  failed  to  cope  up  with  the  underlying  nonlinear  dynamics  for 
short-term  (1,6  and  12)  and  for  long-term  (18  and  24)  month  ahead  prediction. 

Table  1:  24-month  ahead  Prediction,  Data  Partitioning  for  FTLRNN 


Data  Partitioning 

Testing  Data  Set 

Training 

Testing 

MSE 

NMSE 

R 

10% 

75% 

0.03246 

0.37734 

0.49224 

20% 

65% 

0.03657 

0.58297 

0.43616 

30% 

55% 

0.02280 

0.52882 

0.69373 

40% 

45% 

0.01839 

0.83622 

0.77659 

50% 

35% 

0.04066 

0.41935 

0.53973 
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60% 

25% 

0.03059 

0.54899 

0.71497 

70% 

15% 

0.02834 

0.95150 

0.75528 

80% 

05% 

0.02339 

0.57734 

0.8012 

CONCLUSIONS 

It  is  seen  that  focused  time  lagged  recurrent  neural  network  model  with  gamma  memory  is  able  to  predict  the 
northern  sunspots  chaotic  time  series  quite  well  in  comparison  with  the  Multilayer  preceptron  (MLP)  and  self  organizing 
feature  map  (SOFM).  Static  NN  configuration  such  as  MLP  NN  based  model  and  self  organizing  feature  map  (SOFM) 
network  are  failed  to  cope  up  with  the  underlying  nonlinear  dynamics  of  the  sunspots  chaotic  time  series.  It  is  seen  that 
MSE,  NMSE  of  the  proposed  focused  time  lagged  recurrent  neural  network  (FTLRNN)  dynamic  model  for  testing  data  set 
as  well  as  for  training  data  set  are  significant  better  than  those  of  static  MLP  NN  and  SOFMNN  model.  For  the  12,  18  and 
24  months  ahead  prediction  the  value  of  MSE  and  NMSE  for  the  proposed  FTLRNN  model  is  significantly  improved.  Also 
for  the  proposed  FTLRNN  model  the  output  closely  follows  the  desired  output  for  all  the  months  ahead  prediction  for 
northern  sunspots  time  series  as  shown  in  figures  as  compared  to  the  MLP  and  SOFM.  In  addition  it  is  also  observed  that 
the  correlation  coefficient  of  this  model  for  testing  and  training  are  much  higher  than  MLP  and  self  organizing  feature  map 
(SOFM)  neural  network. 

Applications  of  Anns  as  Forecasting  Tools 

To  improve  accuracy  here  we  use  the  data  to  disciplines  and  the  literature  on  forecasting  using  illustrate  a method 
done  to  date  in  the  area.  In  this  section,  There  is  an  extensive  literature  of  research  activities  in  forecasting  with  cautions  of 
ANNs.  ANNs  have  find  applications.  Then  we  will  discuss  the  research  been  used  for  forecasting  bankruptcy  and  business 
methodology  used  in  the  literature. 

FUTURE  SCOPE 

It  is  resulted  from  the  experiments  that  the  FTLRNN  model  learns  thedynamics  of  northern  monthly  sunspot 
chaotic  time  series  quite  well  as  compared  to  Multilayer  perceptron  and  self  organizing  feature  map.  On  the  contrary,  it  is 
observed  that  static  MLP  NN  and  self  organizing  feature  map  (SOFM)  performs  poorly  bad,  because  on  the  one  hand  it 
yields  much  higher  MSE  and  NMSE  on  testing  data  sets  and  on  the  other  hand  the  correlation  coefficient  r for  testing  data 
set  is  far  less  than  unity.  This  is  also  confirmed  from  the  desired  output  Vs  actual  output  plots  for  all  the  steps  for  MLP  and 
SOFM  model  as  shown  in  all  figures  are  the  months  ahead  prediction.  Hence  the  focused  time  lagged  recurrent  neural 
network  with  gamma  memory  filter  has  out  performed  the  static  MLP  based  neural  network  and  SOFM  better  for  all  the 
months’  ahead  predictions  for  monthly  the  chaotic  time  series. 
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